Synthesizing audio of a venue

ABSTRACT

Disclosed herein are system, method, and computer program product embodiments for replicating a remote venue in a local venue. An embodiment operates by receiving original audio. Thereafter, the original stream of audio is modified to produce modified audio based on an audio profile unique to a remote venue smaller than the local venue. The audio profile comprises a virtual representation of the remote venue including (i) a virtual audio source corresponding to the local venue&#39;s audio source and configured to produce the original audio and (ii) a virtual reverberation point corresponding to a real-world location in the remote venue and the portion of the local venue. As such, the modified audio is determined based on receipt of the original audio from the reverberation point. Thus, after correlating the modified audio to the local venue&#39;s audio source, the modified audio is provided by the audio source to the portion of the local venue.

CROSS-REFERENCE RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.16/678,811, filed on Nov. 8, 2019, now U.S. Pat. No. 11,202,162, whichclaims priority to U.S. Provisional Application No. 62/923,205, filed onOct. 18, 2019. U.S. patent application Ser. No. 16/678,811, filed onNov. 8, 2019 is incorporated herein by reference in its entirety.

BACKGROUND

A given musical act (e.g., a singer such as Celine Dion or a band suchas Green Day) will typically perform two or more times at the same venuein the same city. For example, each time Green Day returns to New YorkCity, it may perform in Madison Square Garden. Moreover, a band such asPhish might perform for a series of consecutive days at the same venue,such as Madison Square Garden. Although such musical acts will oftenchange the set list of songs played each night, fans might be lessinclined to see the same act perform multiple times at the same venue.Rather, fans planning to see the same act perform multiple times mightstay away from attending multiple performances at the same venue andmight prefer to see the act play in different venues. Unfortunately,this might not be possible. Either because there are not multiple venuesin the same city which can accommodate the act, or because traveling todifferent venues might be prohibitive for the fans. Instead, what isneeded is a way for a musical act to perform multiple times at the samevenue, but the performances to appear to fans as if it they were beingperformed in multiple different venues.

BRIEF DESCRIPTION OF THE FIGURES

The accompanying drawings are incorporated herein and form a part of thespecification.

FIG. 1 illustrates a block diagram of a system for providing arepresentation of a remote venue in a local venue, according to someembodiments.

FIGS. 2A-B illustrate an example local venue of FIG. 1 , according tosome embodiments.

FIGS. 3A-B illustrates an example remote venue capable of beingrepresented in the local venue of FIG. 1 , according to someembodiments.

FIG. 4 illustrates a method of acquiring a virtual representation of theremote venue of FIGS. 3A-B, according to some embodiments.

FIG. 5 illustrates the virtual representation derived from the method ofFIG. 4 , according to some embodiments.

FIG. 6 illustrates an example local server of FIG. 1 , according to someembodiments.

FIG. 7 illustrates an example block diagram of the components of thelocal server of FIG. 6 that provide audio and video of a remote venue tothe local venue of FIGS. 2A-B, according to some embodiments.

FIG. 8 illustrates a block diagram of an alternative system forproviding a representation of a remote venue in a local venue, accordingto some embodiments.

FIG. 9 illustrates a flowchart of an example method for synthesizingaudio of a venue, according to some embodiments.

FIG. 10 illustrates an example computer system useful for implementingvarious embodiments.

In the drawings, like reference numbers generally indicate identical orsimilar elements. Additionally, generally, the left-most digit(s) of areference number identifies the drawing in which the reference numberfirst appears.

DETAILED DESCRIPTION OF THE INVENTION

Provided herein are system, apparatus, device, method and/or computerprogram product embodiments, and/or combinations and sub-combinationsthereof, for providing a virtual representation of a remote venue in alocal venue.

The present disclosure is directed to permitting the selection of aremote venue to be replicated visually and acoustically in a localvenue. For example, a performer may virtually perform a world tour indifferent cities at the same venue. As such, the performer may select toperform at Red Rocks one night and at St. John the Devine the followingnight. Thus, this disclosure enables continuously representing variousremote venues in a local venue while providing audio and/or video trueto the remote venues in the local venue.

The present disclosure incorporates by reference concurrently filed U.S.patent application Ser. No. 16/678,792, titled “Mapping Audio to VisualImages on a Curved Display Device,” which features may be combined withthose of the present disclosure for synchronously presenting audio of aremote venue with images provides on a three-dimensional displace in alocal venue.

FIG. 1 illustrates a system 100 for providing an audio and videorepresentation of a remote venue in local venues 108A-B. System 100includes backend server 102 in communication with local servers 104A-Band user devices 106A-B. As will be discussed in more detail below,backend server 102 creates a virtual representation of a remote venuefor providing local venues 108A-B with a visual and audio representationof the remote venue. Backend server 102 sends the virtual representationof the remote venue to local servers 104A-B. Local servers 104A-B managevideo and audio reproduction of the remote venue in the local venues108A-B. In some embodiments, although not illustrated, backend server102 and local servers 104A-B may be a single server that performs theircombined functions.

Backend server 102 includes a local venue receiver 110, a remote venuelibrary 112, a spatialization engine 114, and convolution engine 116.Local venue receiver 110 stores dimensions of an interior portion of thelocal venues 108A-B providing the audio presentation, a configurationand location of audio sources 120A-B in local venues 108A-B, a locationof a stage (or where a performer typically performs) in local venues108A-B, and/or a location of possible attendees (e.g., seats) in localvenues 108A-B. As such, local venue receiver 110 may store a distancebetween audio sources 120A-B in the local venues 108A-B, a distance fromeach audio source 120A-B to the stage in the local venue 108A-B.

FIGS. 2A-B illustrate an example local venue 200. As illustrated in FIG.2A, the local venue 200 may be shaped as a dome. As such, as shown inFIG. 2B, the local venue 200 may have an interior surface for providingaudio sources 204A-E to provide audio. The interior surface may alsoprovide display devices 206 to provide images. The display devices 206may follow the shape of the interior surface and thus may also becurved. As such, the display devices 206 may provide a curved displayand partially or surrounds local venue 200's attendees or seats. Indoing so, the display devices 206 may partially surround venue 200'sattendees or seats to provide them with an immersive view of imagerenderings. This may provide viewers with a full 360×180 view of theremote venue. The local venue 200 may include audio sources 204F-G,which may be arranged in an array and correspond to different portionsof the image displayed via the display devices 206. The local venue 200may further include a stage 204 for performers to perform. The backendserver 102 and/or local servers 104A-B (of FIG. 1 ) may determine andutilize a middle point 210 of stage 204.

Referring back to FIG. 1 , backend server 102's remote venue library 112stores various remote venues that can be presented in the local venues108A-B. The remote venue may be a real-world location or a fictitiouslocation. The remote venue may be an outdoor venue or indoor value. Assuch, the remote venue may be a traditional venue that holds events(e.g., Times Square, Red Rocks Amphitheatre, The Gorge Amphitheatre,Hollywood Bowl, and Telluride park) or an untraditional venue thattypically does not hold events (e.g., a beach, a park). Along theselines, the remote venue may be an atypical location for holding events,such as a location that does not permit events or people to enter. Someatypical locations include the Coliseum or Sistine Chapel in Rome,Italy, or Buckingham Palace in London, England, just to provide a fewexamples. Moreover, the remote venue may be smaller or larger than thelocal venue.

FIGS. 3A-B illustrates an example remote venue 300. As illustrated inFIG. 3A, remote venue 300 may be a church and thus may be anuntraditional venue that does not hold events. Thus, as shown in FIG.3B, remote venue 300 may have an interior area 302 for presenting audioand/or video that has an interior surface 302 having a different shapeand/or size than the interior surface of the local venue 200 (of FIG.2B). Along these lines, remote venue 300 may also be composed of adifferent structure (e.g., concrete, wood, and stucco) than the localvenue 200 (of FIG. 2B). Further, remote venue 300 may be a differentgeographical location than local venue 200, which may result indifferent environmental considerations (e.g., humidity, weathercondition, noise pollution, etc.). Any of the aforementioned factors mayaffect the reflectivity of audio waves provided in the remote venue 300and thus affect the perception of audio at remote venue 300.

As such, prior systems were unable to accurately represent audio inremote venues 300 somewhat different than local venues 108A-B (of FIG. 1). In an attempt to do so, prior systems would acquire a single acousticsample of the remote venue 300, typically at the center of the remotevenue 300, and attempt to produce audio throughout the local venue 108Abased on the single acoustic sample of the remote venue 300. However,prior systems did not account for the aforementioned factors unique tothe remote venue. For example, prior systems did not account for thefact that audio waves may reflect in remote venue 300 differently. Andfor this reason, people at different locations of remote venue 300 mayperceive audio differently.

To overcome these challenges, referring now to FIG. 1 , backend server102's spatialization engine 114 stores or creates a virtualrepresentation of remote venue 300's interior area 302 (of FIG. 3B).FIG. 4 illustrates a method for acquiring the virtual representation ofremote venue 400. The virtual representation includes an image portionand an audio portion. To derive the virtual representation's virtualportion, optical device 402 acquires images of remote venue 400. Opticaldevice 402 may be a camera (e.g., a multi-camera array) and acquiresingle still images. As such, images may be acquired from a single pointof view or multiple points of view.

Along these lines, optical device 402 may record a video of images toreplayed on local venue 200's display devices 206 (of FIG. 2B). Forexample, the optical device 402 may record a video of a beach having anocean with splashing waves. This can be provided in the local venue 200to provide the perception that they are on the beach.

As discussed above, local venue 200's display devices 206 (of FIG. 2B)may provide a 360-degree perception of remote venue 400. As such, imagesmay be acquired from different angles and locations to permit anattendee at each possible location in local venue 200 to view anappropriation portion of the remote venue 400. For example, if oneattendee is in the back corner and another attendee is in the front,opposing corner of remote venue 400, the attendee in the back cornerwill have a different view than the attendee in the front, opposingcorner of remote venue 400.

Although remote venue 400 is a building having a closed interior, remotevenue 400 may be completely in the open (e.g., on a beach) or partiallyexposed to the outside (e.g., without a roof). Accordingly, where remotevenue 400 is partially exposed to the outside, images are acquired atall locations and angles such that all attendees in local venue 200 (ofFIG. 2B) view the appropriate portions of remote venue 400 and theoutside. And, where remote venue 400 is completely in the open, a userof user der devices 106A-B first defines an area to acquire thefingerprint. Thereafter, images are acquired at all locations and anglessuch that all attendees in local venue 200 (of FIG. 2A-B) view theappropriate portions of remote venue.

To acquire the virtual representation's audio portion, backend server102's spatialization engine 114 (of FIG. 1 ) determines reverberationsat various locations throughout the remote venue 400. To do so,spatialization engine 114 derives an origin point 414, reflection points406, and reverberation points 408A-L in remote venue 400 to determinereverberations. Origin point 414 corresponds to middle point 210 oflocal venue 200's stage 208 (of FIG. 2B) or another predetermined pointin local venue 200. Audio source 404 is then placed origin point 414.Audio generator 310 may be a pistol or a form of audio, just to providean example.

Reflections points 406A-H are points in the remote venue 400 thatreflect audio. As such, an authorized individual may select reflectionpoints. This may be based on the interior region of the remote venue.For example, if the remote venue has a unique interior portion, areflection point 406A-H may be provided thereon. Moreover, reflectionpoints 406 may be based on the location of audio sources 204A-E in localvenue 200 (of FIG. 2B). As such, the locations of reverberation points408A-L in the remote venue 400 may correspond to locations of audiosources 204A-E in local venue 200. In turn, the number reverberationpoints 408A-L in remote venue 400 may correspond to the number of audiosources 204A-E in local venue 200. Further, an optimal number ofreflection points 406 may be selected based on the location and/or thesize of the local venue.

Reverberation points 408A-L are placed a distance away from reflectionpoints 406A-H. An authorized individual may preselect the distance,which may be based on the size of the remote venue. In some embodiments,where the remote venue 400 is smaller than the local venue 200 (of FIG.2B), reverberation points 408A-L in remote venue 400 may be less thanthe number of audio sources 204A-E (of FIG. 2B) in the local venue 200.Further, where the remote venue 400 is smaller than the local venue 200,a single reverberation point 408A-L of remote venue 400 may correspondto multiple audio sources 204A-e. Moreover, where the remote venue 400is larger than local venue 200, reverberation points 408A-L of remotevenue 400 may be greater than the number of audio sources 204A-E inlocal venue 200.

Accordingly, first audio receivers 410A-H (e.g., microphones) are placedat or near reflection points 406A-H in remote venue 400. And, secondaudio receivers 412A-L (e.g., microphones) are placed at or nearreverberation points 408A-L in remote venue 400. Audio source 404 thenprovides audio and audio receivers 410A-H/412A-L at reflection andreverberation points 406A-H/408A-L receives the audio. As such, for eachrespective reflection point 406A-H, the associated audio receivers410A-H/412A-L may receive audio from audio source 404.

In some embodiments, the effect of the reflection points 406A-H may bedetected collectively. For example, first and second audio receivers410A-H/412A-L may receive the audio provided by audio source 404 at thesame time. In some embodiments, the effect of the reflection points406A-H may be determined individually. For example, for reflection point406A, associated first and second audio receivers 410A/412A receiveaudio from audio source 404. Thereafter, for each reflection point406B-H, associated first and second audio receivers 410B-H/412B-Hreceived audio from audio source 404 sequentially. However, as discussedabove, depending on the size of the remote venue with respect to thelocal venue, additional second audio receivers 412A-L may correspond tothe same reflection point 406A or first audio receiver 412A-L.

Along these lines, audio source 404 may provide audio in multiplepredetermined directions in remote venue 400. For example, when theeffect of the reflection points 406A-H are determined collectively andthus first and second audio receivers 410A-H/412A-L receives audio fromaudio source 404 at the same time, audio source 404 may be directedsequentially at each of the reflection points 406A-H. Alternatively,when the effect of the reflection points 406A-H is determinedindividually and variously associated first and second audio receivers410A/412A receive audio from audio source 404 at different times, theaudio source 404 may be directed at the reflection point 406A-H beanalyzed. For example, if reflection point 406A is being analyzed, theaudio source 404 may be directed at reflection point 406A. As such, thismay be performed sequentially for each of the remaining reflectionpoints 406B-H.

Moreover, audio source 404 may provide audio at different sound pressurelevels (SPLs) (e.g., decibels) within a predetermined frequency range(e.g., from 20 Hz to 20 kHz). Furthermore, audio source 404 may be movedto different locations in the remote venue 400. As such, audio receivers410A-H/412A-L at reflection points 406A-H and reverberation points408A-L throughout remote venue 400 may receive audio from an audiosource 404 at different locations and different SPLs.

Along these lines, while deriving the reverberations, environmentalaudios unique to the remote venue may be derived and recorded at thereverberation points 408A-L. Examples of environmental audio includewaves breaking and seagulls at a beach, echoing footsteps in a church,and wind and rustling leaves in the outdoors, just to provide a fewexamples. The environmental audio may be mixed with the modified streamof audio and may be provided at a frequency lower than the modifiedstream of audio.

The spatialization engine 114 (of FIG. 1 ) then determines or measuresreverberations at each reverberation point 408A-L in remote venue 400and SPL provided by audio source 404. As understood by a person ofordinary skill in the art, reverberation refers to the persistence ofaudio after the audio is produced and includes a time required for theaudio to fade away. Reverberation time of the remote venue 400 is basedon real-world characteristics of remote venue 400, such as anarchitectural design (e.g., size, shape, and materials) of remote venue400 and objects (e.g., water, mountains, furniture, building structure)in or around remote venue 400, just to provide a few examples. As such,the reverberations in remote venue 400 are unique to remote venue 400.Thus, the reverberations for the reverberation points may includedifferent delays and/or frequency-dependent level variations, dependingon the remote venue 400 and the location in the remote venue 400.

The spatialization engine 114 (of FIG. 1 ) also may create a virtualrepresentation of remote venue 400, optionally after deriving thereverberations for the reverberation points 408A-L. As the remote venueis three-dimensional, the virtual representation may also bethree-dimensional. As such, where remote venue 400 is an indoor venuehaving structural portions, the virtual presentation may depict variousstructural portions of the indoor venue. In turn, some of thesestructural portions of remote venue 400 may be different thancorresponding structural portions of local venue 108A-B (of FIG. 1 ).Thus, reflection points 406A-H and second audio receiver 412A-L may beprovided at structural portions of remote venue 400 different thancorresponding structural portions of local venues 108A-B.

FIG. 5 illustrates an example virtual representation 500 of remote venue400 (of FIG. 4 ). Virtual representation 500 includes virtualreverberation points 502A-L, virtual speakers 504A-H, a virtual originpoint 506, and a virtual audio source 508. Virtual reverberation points502A-L correspond to reverberation points 408A-L (of FIG. 4 ) andreverberations determined by spatialization engine 114 (of FIG. 4 )relating to the reverberation points 408A-L as discussed above. As such,virtual reverberation points 502A-L may have a three-dimensionalposition in the virtual representation corresponding to thereverberation points 408A-L in the remote venue 400.

Virtual audio sources 504A-L correspond to local venue 200's audiosources 204A-E (of FIG. 2B). Although not illustrated in these figures,the number of virtual audio sources 504A-L equals the number of audiosources 204A-E. As such, the virtual audio sources 504A-L may have athree-dimensional location in the virtual representation 500 of theremote venue that corresponds to the three-dimensional location of thelocal venue 200's audio sources 204A-E. For example, the virtual audiosources 504A-L may have the same spatial relationship with respect tothe virtual origin point 506 in the virtual representation as to theaudio sources 204A-E with respect to the middle point 210 of local venue200's stage 208 (of FIG. 2B) or predetermined point in local venue 200.

Virtual origin point 506 corresponds to the same middle point 210 oflocal venue 200's stage 208 (of FIG. 2B), or the same predeterminedpoint in local venue 200, as origin point 414 (of FIG. 4 ). Virtualaudio source 508 corresponds to an audio source provided in local venue200, which may be part of a prerecorded video or a live performer.Accordingly, virtual audio source 508 moves in the virtualrepresentation based on the movement of the audio source in theprerecorded video or live performance.

As such, the virtual audio source 508 may transverse and rotate in thevirtual representation 500 based on the movement of the audio source120A-B in the local venue 108A-B (of FIG. 1 ). Thus, the local server104A-B may track and send the movement of the audio source 120A-B to thebackend server 102 (of FIG. 1 ). For example, if the audio source 120A-Bmoves left to right in the local venue 104A-B, the audio source 508 maymove accordingly in the virtual representation. Moreover, if the audiosource rotates 30 degrees in a certain direction in the local venue, theaudio source 508 may rotate accordingly in the virtual representation.

Along these lines, the virtual audio source 508 may change size in thevirtual representation 500 based on a degree of focus of a video camerarecording images of the audio source of the local venue. As such, basedon the size of the virtual audio source 508 in the virtualrepresentation 500, the audio reproduced by the virtual audio source 508may be modified accordingly. For example, if the video camera focuses onthe audio source such that the audio source is depicted at two times anormal magnification level, the virtual audio source 508 may be doubledin size and reproduce audio twice as loud, relative to the other audiosource, as normal in the virtual representation 500. Conversely, if thevideo camera expands the field of view such that the magnification levelis half of the normal magnification level, the virtual audio source 508may be half of its normal size and reproduced audio half as loud,relative to the other audio source, as normal in the virtualrepresentation 500.

As such, the virtual representation 500 incorporates reverberationpoints 502A-L determined in the real-world remote venue 400 (of FIG. 4 )and replications of properties of the audio sources in the local venue200 (of FIG. 2B).

Referring back to FIG. 1 , local servers 104A-B may provide the audioand video to the local venue 108A-B's display devices 118A-B and audiosources 120A-B, respectively. FIG. 6 illustrates an example local server600 of the local venues 108A-B (of FIG. 1 ). Local server 600 includesthe local venue receiver 602, remote venue library 604, audio mixer 606,audio server 608, and video server 610. Like backend server 102's localvenue receiver 110 (of FIG. 1 ), local venue receiver 602 storesdimensions of an interior portion of the local venues 108A-B providingthe audio presentation, a configuration and location of audio sources120A-B in local venues 108A-B (of FIG. 1 ), a location of a stage (orwhere a performer typically performs) in local venues 108A-B, and/or alocation of possible attendees (e.g., seats) in local venues 108A-B. Assuch, local venue receiver 602 may store a distance between audiosources 112A-B in the local venues 108A-B, a distance from each audiosource 120A-B to the stage in the local venue 108-B.

Also like backend server 102's remote venue library 112 (of FIG. 1 ),remote venue library 604 stores audio and video of various remote venuesthat can be presented in the local venues 108A-B (of FIG. 1 ). Remotevenue library 604 also stores virtual representations for the remotevenues.

Audio mixer 606 receives live audio from a local venue. The audio mixer606 may receive prerecorded audio from a video provided in the localvenue 200. Alternatively, the audio mixer 606 may receive audio from alive performer in the local venue 200. In some embodiments, as describedabove, the audio mixer 606 receives audio from multiple live performersin the local venue.

Audio server 608 comprises audio effect filters 612 to mirror thereverberation points of the remote venues. The audio effect filter 612may be based on the virtual representation of the remote venues.Accordingly, the audio effect filters 612 identify reverberations fordifferent portions of the remote venue based on the correspondingreverberation points in the virtual representation of the remote venue.The audio effect filters 612 then applies the reverberations of thedifferent portions of the remote portion so that audio having thereverberations can be replayed in corresponding portions the localvenue, thereby allowing the local venue to replicate the remote venueacoustically.

Video server 610 receives images or video of the remote venue from theremote venue library 604 and/or from the local venue.

FIG. 7 illustrates an example block diagram of the components of thelocal server 600 (of FIG. 6 ) providing audio and video of a remotevenue to local venue 200 (of FIGS. 2A-B). As described above, localvenue 200 may be shaped as a dome. As such, the local venue 200 may havean interior surface for providing audio sources 204A-E to provide audio.The interior surface may also provide display devices 206 to provideimages. The display devices 206 may follow the shape of the interiorsurface and thus may also be curved. As such, the display devices 206may provide a curved display and partially or surrounds local venue200's attendees or seats, thereby providing them with an immersive viewof image renderings. This may provide viewers with a full 360×180 degreeview of the remote venue. The local venue 200 may include audio sources204F-G, which may be arranged in an array and correspond to differentportions of the images displayed via the display devices 206 (e.g.,images depicted of a remote venue or provided onto those of the remotevenue and belonging to the local venue). The local venue 200 may furtherinclude a stage 204 for performers to perform.

As such, audio server 608 receives the virtual representation of theremote venue 200 from the remote venue library 604. Audio server 608derives audio effect filters 612 based on the reverberation points inthe virtual representation of the remote venue 200.

Audio server 608 also receives audio from audio mixer 606, whichreceives audio from local venue 200. Audio server 608 determinesreverberations for audio sources 204A-G of the local venue based on theaudio effect filters 612 of the remote venue. As such, audio sources204A-G of the local venue 200 may provide audio to attendees replicatingaudio that the attendees would hear in corresponding areas in the remotevenue. For example, if audio sources 204A/E provide audio to a first andsecond area of local venue 200 that corresponds to a first and secondarea of the remote venue, audio sources 204A/204E provide modified audioto the first and second portions of local venue 200 havingreverberations. The modified audio corresponds to audio provided in thefirst and second portions of the remote venue. Along these lines, theaudio sources 204A/204E may provide different reverberations.

As stated above, video server 610 receives images of the remote venuefrom the remote venue library 604 and/or images from the local venue200. Video server 610 modifies images of the remote venue and/or oflocal venue based on the shape of the display devices 206 of the localvenue 200 to provide the attendees of the local venue 200 with anaccurate representation of the remote venue.

As such, the video server 610's images may include audio sources (e.g.,live performers or of a movie) in local venue 200. The video server610's images may also be part of a prerecorded video (e.g., a movie)provided in the local venue. Accordingly, the video server 610's imagesmay depict an audio source moving and continually modify the images topresent the visual representation of the audio source more accurately.

Along these lines, the video server 610 provides different images of theremote venue to the appropriate portions (e.g., a first and secondportion) of the local venue 200's display devices 206 so that anaccurate visual representation of the remote venue is provided in thelocal venue. And, as noted above, the audio server 608 providesdifferent modified portions of the original audio (e.g., a first andsecond modified portion) to the appropriate local venue 200's audiosources 204A-G so that an accurate acoustic representation is providedin the local venue 200. Accordingly, the audio and video servers 608/610may send the images and modified streams of audio synchronously, forexample, by being based on a time sequence.

Referring back to FIG. 1 , as discussed above, backend server 102 mayfurther include the convolution engine 116. Convolution engine 116performs the functions of the local server 104A-B. As such, convolutionengine 116 may determine a view of images presented by the displaydevices 118 of the local venues 108A-B at various locations in the localvenues 108A-B. Similarly, convolution engine 116 may determine alocation of audio provided by audio sources 120A-B of the local venues108A-B at various locations in the local venues 108A-B.

As such, via backend server 102's convolution engine 116 and userdevices 106A-B permit authorized users to review and/or preview imageryand/or audio at different locations in the local venues 108A-B. Alongthese lines, user devices 106A-B may also permit simulation and/ormodeling of remote locations to be presented in local venues 108A-B. Thesimulation may permit an authorized user to view and/or edit theprocessing performed by the convolution engine 116 in creating thereplication of the remote venue 300. As such, the simulation may permitan authorized user to edit one or more inputs of the convolution engine,such as reverberations of audio in the virtual representation. Moreover,the modeling may permit an authorized user to view imagery and/or listento audio from predefined points in the venue. The authorized users maythen make any suitable modifications to the imagery and/or audio foreach predefined point. For example, the authorized user may modify thereverberation of audio, the location of imagery on display devices,and/or mapping of the audio to the imagery. Thus, user devices 106A-Bmay be in the form of headphones, a display device, and/or a virtualreality headset, just to name a few examples.

FIG. 8 illustrates an alternative system 800 for providing an audio andvideo representation of a remote venue in a local venue 804A-B. System800 includes backend server 802 in communication local venues 804A-B anduser devices 806A-B. In contrast to system 100 of FIG. 1 , system 800does not include local servers 104A-B. Rather, backend server 802 actsas a single server for backend server 102 and local servers 104A-B (ofFIG. 1 ). As such, like backend server 102 (of FIG. 1A), backend server802 includes local venue receiver 808, remote venue library 810,spatialization engine 812, and convolution engine 814. However, inaddition, backend server 802 also includes the additional components oflocal server 600 (of FIG. 6 ). As such, backend server 102 furtherincludes audio mixer 816, audio server 818, and video server 820.

FIG. 9 illustrates a flowchart of a method of replicating a remote venuein a local venue, according to some embodiments. Methods 900 can beperformed by processing logic that can comprise hardware (e.g.,circuitry, dedicated logic, programmable logic, microcode, etc.),software (e.g., instructions executed on a processing device), or acombination thereof. It is to be appreciated that not all steps may beneeded to perform the disclosure provided herein. Further, some of thesteps may be performed simultaneously or in a different order than shownin FIG. 9 , as will be understood by a person of ordinary skill in theart.

Referring now to FIG. 9 , method 900 shall be described with referenceto FIGS. 1 and 3A-B. However, method 900 is not limited to those exampleembodiments.

In 902, the backend server 102 receives an original stream of audio at alocal venue 108A. The local venue comprises an audio source 120Aconfigured to provide audio to a portion of the local venue 108A.

In 904, the backend server 102 modifies the original stream of audio toproduce a modified stream of audio based on an audio profile unique to aremote venue 300 smaller than the local venue 108A.

The audio profile comprises a virtual representation of the remotevenue, which comprises (i) a virtual audio source corresponding to theaudio source of the local venue and configured to produce the originalstream of audio and (ii) a virtual reverberation point corresponding toa real-world location in the remote venue and the portion of the localvenue. The modified stream of audio is determined based on receipt ofthe original stream of audio from the reverberation point

In 906, the backend server 102 correlates the modified stream of audioto the audio source 120A of the local venue 108A such that the stream ofimages are provided in sync with the modified stream of audio.

In 908, the backend server 102 presents the modified stream of audio onthe audio source 120A to the portion of the local venue 108A

Various embodiments may be implemented, for example, using one or morewell-known computer systems, such as computer system 1000 shown in FIG.10 . One or more computer systems 1000 may be used, for example, toimplement any of the embodiments discussed herein, as well ascombinations and sub-combinations thereof.

Computer system 1000 may include one or more processors (also calledcentral processing units, or CPUs), such as a processor 1004. Processor1004 may be connected to a communication infrastructure or bus 1006.

Computer system 1000 may also include user input/output device(s) 1003,such as monitors, keyboards, pointing devices, etc., which maycommunicate with communication infrastructure 1006 through userinput/output interface(s) 1002.

One or more processors 1004 may be a graphics processing unit (GPU). Inan embodiment, a GPU may be a processor that is a specialized electroniccircuit designed to process mathematically intensive applications. TheGPU may have a parallel structure that is efficient for processing oflarge blocks of data simultaneously (in parallel as opposed toserially), such as mathematically intensive data common to computergraphics applications, images, videos, etc.

Computer system 1000 may also include a main or primary memory 1008,such as random access memory (RAM). Main memory 1008 may include one ormore levels of cache. Main memory 1008 may have stored therein controllogic (i.e., computer software) and/or data.

Computer system 1000 may also include one or more secondary storagedevices or memory 1010. Secondary memory 1010 may include, for example,a hard disk drive 1012 and/or a removable storage device or drive 1014.Removable storage drive 1014 may be a floppy disk drive, a magnetic tapedrive, a compact disk drive, an optical storage device, tape backupdevice, and/or any other storage device/drive.

Removable storage drive 1014 may interact with a removable storage unit1018. Removable storage unit 1018 may include a computer-usable orreadable storage device having stored thereon computer software (controllogic) and/or data. Removable storage unit 1018 may be a floppy disk,magnetic tape, compact disk, DVD, optical storage disk, and/any othercomputer data storage device. Removable storage drive 1014 may read fromand/or write to a removable storage unit 1018.

Secondary memory 1010 may include other means, devices, components,instrumentalities or other approaches for allowing computer programsand/or other instructions and/or data to be accessed by the computersystem 1000. Such means, devices, components, instrumentalities or otherapproaches may include, for example, a removable storage unit 1022 andan interface 1020. Examples of the removable storage unit 1022 and theinterface 1020 may include a program cartridge and cartridge interface(such as that found in video game devices), a removable memory chip(such as an EPROM or PROM) and associated socket, a memory stick and USBport, a memory card and associated memory card slot, and/or any otherremovable storage unit and associated interface.

Computer system 1000 may further include a communication or networkinterface 1024. Communication interface 1024 may enable computer system1000 to communicate and interact with any combination of externaldevices, external networks, external entities, etc. (individually andcollectively referenced by reference number 1028). For example,communication interface 1024 may allow computer system 1000 tocommunicate with external or remote devices 1028 over communicationspath 1026, which may be wired and/or wireless (or a combinationthereof), and which may include any combination of LANs, WANs, theInternet, etc. Control logic and/or data may be transmitted to and fromcomputer system 1000 via communication path 1026.

Computer system 1000 may also be any of a personal digital assistant(PDA), desktop workstation, laptop or notebook computer, netbook,tablet, smartphone, smartwatch or another wearable, appliance, part ofthe Internet-of-Things, and/or embedded system, to name a fewnon-limiting examples, or any combination thereof.

Computer system 1000 may be a client or server, accessing or hosting anyapplications and/or data through any delivery paradigm, including butnot limited to remote or distributed cloud computing solutions; local oron-premises software (“on-premise” cloud-based solutions); “as aservice” models (e.g., content as a service (CaaS), digital content as aservice (DCaaS), software as a service (SaaS), managed software as aservice (MSaaS), platform as a service (PaaS), desktop as a service(DaaS), framework as a service (FaaS), backend as a service (BaaS),mobile backend as a service (MBaaS), infrastructure as a service (IaaS),etc.); and/or a hybrid model including any combination of the foregoingexamples or other services or delivery paradigms.

Any applicable data structures, file formats, and schemas in computersystem 1000 may be derived either extemporaneously or from standardsincluding but not limited to JavaScript Object Notation (JSON),Extensible Markup Language (XML), Yet Another Markup Language (YAML),Extensible Hypertext Markup Language (XHTML), Wireless Markup Language(WML), MessagePack, XML User Interface Language (XUL), or any otherfunctionally similar representations alone or in combination.Alternatively, proprietary data structures, formats, or schemas may beused, either exclusively or in combination with known or open standards.

In some embodiments, a tangible, non-transitory apparatus or article ofmanufacture comprising a tangible, non-transitory computer useable orreadable medium having control logic (software) stored thereon may alsobe referred to herein as a computer program product or program storagedevice. This includes, but is not limited to, computer system 1000, mainmemory 1008, secondary memory 1010, and removable storage units 1018 and1022, as well as tangible articles of manufacture embodying anycombination of the foregoing. Such control logic, when executed by oneor more data processing devices (such as computer system 1000), maycause such data processing devices to operate as described herein.

Based on the teachings contained in this disclosure, it will be apparentto persons skilled in the relevant art(s) how to make and useembodiments of this disclosure using data processing devices, computersystems and/or computer architectures other than that shown in FIG. 10 .In particular, embodiments can operate with software, hardware, and/oroperating system implementations other than those described herein.

It is to be appreciated that the Detailed Description section, and notany other section, is intended to be used to interpret the claims. Othersections can set forth one or more but not all exemplary embodiments ascontemplated by the inventor(s), and thus, are not intended to limitthis disclosure or the appended claims in any way.

While this disclosure describes exemplary embodiments for exemplaryfields and applications, it should be understood that the disclosure isnot limited thereto. Other embodiments and modifications thereto arepossible and are within the scope and spirit of this disclosure. Forexample, and without limiting the generality of this paragraph,embodiments are not limited to the software, hardware, firmware, and/orentities illustrated in the figures and/or described herein. Further,embodiments (whether or not explicitly described herein) havesignificant utility to fields and applications beyond the examplesdescribed herein.

Embodiments have been described herein with the aid of functionalbuilding blocks illustrating the implementation of specified functionsand relationships thereof. The boundaries of these functional buildingblocks have been arbitrarily defined herein for the convenience of thedescription. Alternate boundaries can be defined as long as thespecified functions and relationships (or equivalents thereof) areappropriately performed. Also, alternative embodiments can performfunctional blocks, steps, operations, methods, etc. using orderingsdifferent than those described herein.

References herein to “one embodiment,” “an embodiment,” “an exampleembodiment,” or similar phrases, indicate that the embodiment describedcan include a particular feature, structure, or characteristic, butevery embodiment can not necessarily include the particular feature,structure, or characteristic. Moreover, such phrases are not necessarilyreferring to the same embodiment. Further, when a particular feature,structure, or characteristic is described in connection with anembodiment, it would be within the knowledge of persons skilled in therelevant art(s) to incorporate such feature, structure, orcharacteristic into other embodiments whether or not explicitlymentioned or described herein. Additionally, some embodiments can bedescribed using the expression “coupled” and “connected” along withtheir derivatives. These terms are not necessarily intended as synonymsfor each other. For example, some embodiments can be described using theterms “connected” and/or “coupled” to indicate that two or more elementsare in direct or electrical contact with each other. The term “coupled,”however, can also mean that two or more elements are not in directcontact with each other, but yet still co-operate or interact with eachother.

The breadth and scope of this disclosure should not be limited by any ofthe above-described exemplary embodiments but should be defined only inaccordance with the following claims and their equivalents.

What is claimed is:
 1. A method for developing an audio profile forsound traveling through a venue, the method comprising: identifying, bya computer system, a first plurality of locations of a plurality ofreflection points within the venue, the first plurality of locationsbeing based on a second plurality of locations of a plurality of audiosources within a second venue that is different from the venue;identifying, by the computer system, a plurality of reverberation pointswithin the venue, each reverberation point from among the plurality ofreverberation points being a distance away from a correspondingreflection point from among the plurality of reflection points;directing the sound from a first location within the venue toward asecond location from among the first plurality of locations of a firstreflection point from among the plurality of reflection points; andanalyzing, by the computer system, the sound as received at the firstreflection point and a first reverberation point from among theplurality of reverberation points that corresponds to the firstreflection point to develop the audio profile for the venue.
 2. Themethod of claim 1, further comprising: determining, by the computersystem, an origin point with the venue that corresponds to a middlepoint of a stage within the venue, and wherein the first location withinthe venue comprises the origin point.
 3. The method of claim 1, whereinthe identifying the plurality of reverberation points comprises:identifying a third plurality of locations of the plurality ofreverberation points within the venue, the third plurality of locationscorresponding to the second plurality of locations of the plurality ofaudio sources within the second venue, wherein a number of reverberationpoints from among the plurality of reverberation points is less than anumber of audio sources from among the plurality of audio sources whenthe second venue is smaller than the venue, and wherein the number ofreverberation points from among the plurality of reverberation points isgreater than the number of audio sources when the second venue is largerthan the venue.
 4. The method of claim 1, wherein the analyzingcomprises: analyzing, by the computer system, the sound as received at athird location from among the first plurality of locations of a secondreflection point from among the plurality of reflection points and asecond reverberation point that corresponds to the second reflectionpoint from among the plurality of reverberation points to develop theaudio profile for the venue.
 5. The method of claim 1, wherein thedirecting comprises: directing the sound from the first location withinthe venue at a sound pressure level (SPL) selected from a plurality ofSPLs at a frequency within a predetermined frequency range.
 6. Themethod of claim 1, further comprising: directing a second sound from athird location within the venue toward the first reflection point; andanalyzing the second sound as received at the first reflection point andthe first reverberation point to develop the audio profile for thevenue.
 7. The method of claim 1, wherein the analyzing comprises:measuring a reverberation of the sound at the first reverberation point,wherein a duration of the reverberation of the sound is based upon areal-world characteristic of the venue.
 8. A computer system fordeveloping an audio profile for sound traveling through a venue, thecomputer system comprising: a memory that stores instructions; and aprocessor configured to execute the instructions stored in the memory,the instructions, when executed by the processor, configuring theprocessor to: identify a first plurality of locations of a plurality ofreflection points within the venue, the first plurality of locationsbeing based on a second plurality of locations of a plurality of audiosources within a second venue that is different from the venue, identifya plurality of reverberation points within the venue, each reverberationpoint from among the plurality of reverberation points being a distanceaway from a corresponding reflection point from among the plurality ofreflection points, and analyze a sound, directed from a first locationwithin the venue toward a first reflection point from among theplurality of reflection points, as received at the first reflectionpoint and a first reverberation point from among the plurality ofreverberation points that corresponds to the first reflection point todevelop the audio profile for the venue.
 9. The computer system of claim8, wherein the instructions, when executed by the processor, furtherconfigure the processor to: determine an origin point with the venuethat corresponds to a middle point of a stage within the venue, andwherein the first location within the venue comprises the origin point.10. The computer system of claim 8, wherein the instructions, whenexecuted by the processor, configure the processor to identify a thirdplurality of locations of the plurality of reverberation points withinthe venue, the third plurality of locations corresponding to the secondplurality of locations of the plurality of audio sources within thesecond venue, wherein a number of reverberation points from among theplurality of reverberation points is less than a number of audio sourcesfrom among the plurality of audio sources when the second venue issmaller than the venue, and wherein the number of reverberation pointsfrom among the plurality of reverberation points is greater than thenumber of audio sources when the second venue is larger than the venue.11. The method of claim 8, wherein the instructions, when executed bythe processor, further configure the processor to analyze the sound asreceived at a third location from among the first plurality of locationsof a second reflection point from among the plurality of reflectionpoints and a second reverberation point from among the plurality ofreverberation points that corresponds to the second reflection point todevelop the audio profile for the venue.
 12. The computer system ofclaim 8, wherein the sound is configured to be at a sound pressure level(SPL) selected from a plurality of SPLs at a frequency within apredetermined frequency range.
 13. The computer system of claim 8,wherein the instructions, when executed by the processor, furtherconfigure the processor to: analyze a second sound, directed from athird location within the venue toward the first reflection point, asreceived at the first reflection point and the first reverberation pointto develop the audio profile for the venue.
 14. The computer system ofclaim 8, wherein the instructions, when executed by the processor,further configure the processor to measure a reverberation of the soundat the first reverberation point, and wherein a duration of thereverberation of the sound is based upon a real-world characteristic ofthe venue.
 15. A system for developing an audio profile for soundtraveling through a venue, the system comprising: a first plurality ofaudio receivers at a first plurality of locations of a plurality ofreflection points within the venue, the first plurality of locationsbeing based on a second plurality of locations of a plurality of audiosources within a second venue that is different from the venue; a secondplurality of audio receivers at a plurality of reverberation pointswithin the venue, each reverberation point from among the plurality ofreverberation points being a distance away from a correspondingreflection point from among the plurality of reflection points; an audiosource configured to direct the sound from a first location within thevenue toward a second location from among the first plurality oflocations of a first reflection point from among the plurality ofreflection points; and a computer server configured to analyze the soundas received at the first reflection point and a first reverberationpoint from among the plurality of reverberation points that correspondsto the first reflection point to develop the audio profile for thevenue.
 16. The system of claim 15, wherein the plurality ofreverberation points are at a third plurality of locations within thevenue, the third plurality of locations corresponding to the secondplurality of locations of the plurality of audio sources within thesecond venue, wherein a number of reverberation points from among theplurality of reverberation points is less than a number of audio sourcesfrom among the plurality of audio sources when the second venue issmaller than the venue, and wherein the number of reverberation pointsfrom among the plurality of reverberation points is greater than thenumber of audio sources when the second venue is larger than the venue.17. The system of claim 15, wherein the computer server is furtherconfigured to analyze the sound as received at a third location fromamong the first plurality of locations of a second reflection point fromamong the plurality of reflection points and a second reverberationpoint from among the plurality of reverberation points that correspondsto the second reflection point to develop the audio profile for thevenue.
 18. The method of claim 1, wherein the venue comprises a remotevenue, and wherein the second venue comprises a local venue that is toreplicate the remote venue.
 19. The computer system of claim 8, whereinthe venue comprises a remote venue, and wherein the second venuecomprises a local venue that is to replicate the remote venue.
 20. Thesystem of claim 15, wherein the venue comprises a remote venue, andwherein the second venue comprises a local venue that is to replicatethe remote venue.